Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(autoware_image_projection_based_fusion): add mask cluster fusion node #9482

Open
wants to merge 20 commits into
base: main
Choose a base branch
from

Conversation

StepTurtle
Copy link
Contributor

@StepTurtle StepTurtle commented Nov 27, 2024

Description

This PR introduces a new node for fusing instance segmentation results with clusters, assigning label values to the clusters. It serves as an alternative to the roi_cluster_fusion node, which relies on bounding boxes.

Following table shows results with old and new approach:

Node Parameters Result Video
roi_cluster_fusion only_allow_inside_cluster = false https://github.com/user-attachments/assets/63fbb81d-5ae5-4d28-8918-1763789afa81
roi_cluster_fusion only_allow_inside_cluster = true https://github.com/user-attachments/assets/c9f966d2-60b6-49b3-bcb4-88ae998be4a2
mask_cluster_fusion default https://github.com/user-attachments/assets/f450af0c-a6af-49d0-9b78-1b089f65f577

Related links

Parent Issue:

How was this PR tested?

Testers can follow these steps to test PR.

1) Download the pre-trained RTMDet model:

gdown https://drive.google.com/drive/folders/1P1JyMrstJIN9Y4F947nUkbKxQ2JDntgF?usp=drive_link -O ~/autoware_data/ --folder

2) For autoware use following PR:

3) For autoware.universe use this and following PR:

4) Build your Autoware workspace with new changes.

5) Run e2e or logging simulator.

6) Update the topic parameters from launch/rtmdet.launch.xml and run the launch file

ros2 launch autoware_tensorrt_rtmdet rtmdet.launch.xml

7) Update the topic parameters from launch/mask_cluster_fusion.launch.xml and run the launch file

ros2 launch autoware_image_projection_based_fusion mask_cluster_fusion.launch.xml
  • You should able to see newly added label names for cluster from /perception/object_recognition/detection/clustering/camera_lidar_fusion/clusters topic.

Notes for reviewers

Interface changes

None.

Effects on system behavior

None.

Signed-off-by: Barış Zeren <[email protected]>
Signed-off-by: Barış Zeren <[email protected]>
Signed-off-by: Barış Zeren <[email protected]>
Signed-off-by: Barış Zeren <[email protected]>
@StepTurtle StepTurtle added the component:perception Advanced sensor data processing and environment understanding. (auto-assigned) label Nov 27, 2024
@StepTurtle StepTurtle self-assigned this Nov 27, 2024
Copy link

github-actions bot commented Nov 27, 2024

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

Copy link
Contributor

@kminoda kminoda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR. Would you mind making this PR a draft and re-open it once the documentation is complete (e.g. docs/mask_cluster_fusion_node.md)?

@github-actions github-actions bot added the type:documentation Creating or refining documentation. (auto-assigned) label Dec 5, 2024
@StepTurtle
Copy link
Contributor Author

Thank you for the PR. Would you mind making this PR a draft and re-open it once the documentation is complete (e.g. docs/mask_cluster_fusion_node.md)?

Hey @kminoda, documentation is added.

related commit: 9187a9e

@StepTurtle StepTurtle added the run:deploy-docs Mark for deploy-docs action generation. (used-by-ci) label Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:perception Advanced sensor data processing and environment understanding. (auto-assigned) run:deploy-docs Mark for deploy-docs action generation. (used-by-ci) tag:require-cuda-build-and-test type:documentation Creating or refining documentation. (auto-assigned)
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

2 participants